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Detailed Action 



Drawings 

1 . Drawings were not objected to by the Draftsperson under 37 CFR 1 .84 or 1 . 1 52, 
and stamped approved. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. § 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title; if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1-5, 13-16, 25-27, 31-32 and 37-39 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Hou et. al. (Hou) U.S. Patent No. 5,838,313 in view of Sidana 
U.S. Patent No. 6,081,829. 

Regarding claim 7-3, 25 and 29-30 Hou teaches substantial features of the invention as 
claimed; Hou teaches a system comprising: a client computer (col 6/lines 38-51) to 
playback multimedia content (col 8/lines 1-15, col 6/lines 30-37) and annotations 
corresponding to different temporal portions of the multimedia content (col 6/lines 61-66, 
col 7/lines 1-3, col 8/lines 61-63, col 9/lines 9-14); maintaining an annotation database 
(Fig. 9, 33, col 6/lines 12-14) having a plurality of annotations (col 3/lines 54-col 4/line 
25) corresponding to the multimedia content (col 6/lines 38-41), provide the plurality of 
annotations to the client computer for playback (Fig. 6, playback means, Fig. 12, lines 
56-61, col 6/lines 23-25, display annotations means, col 8/lines 43-48), send electronic 
mail messages (col 3/lines 8-13, create or reply means: col 2/lines 25-37, sending and 
displaying mail: col 11/lines 22-25, creating e-mail message: col 6/lines 14-16) including 
annotations (col 1 1/lines 3-25, Fig. 1) to recipients identified by the client computer (col 
4/lines 26-3 1, Fig. 4), via interface means to receive input data regarding new annotation 
corresponding to media content (col 6/lines 26-51), generate new annotations create or 
reply means (col 2/lines^^-£J,£col 3/lines 9-13) based on received electronic mail 
messages (col 4/lines 10-14, Fig. 2), and add the new annotations (col 9/lines 15-32, col 
1/lines 17-20, adding annotations means: col 5/lines 29-52, report: col 6/lines 10-14, Fig. 
3, saving in database) to the annotation database. 

However Hou does not explicitly teach where coupled to the client computer to 
maintain an annotation database and to manage streaming the multimedia content to the 
client computer is particularly an annotation media server computer, where further an 
identifier of the media content to which the new annotation corresponds. 
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Sidana teaches a system/method for enabling client using an user interface to be 
presented with web-viewable documents including annotations maintaining a annotation 
multimedia server computer coupled to the client for streaming the multimedia to the 
client computer (abstract, col 2/lines 34-59), disclosing a system wherein a client 
computer is provided with annotations corresponding to multimedia content, where 
annotation media server coupled to the client computer is coupled to an annotation server 
via a network (col 4/lines 14-33, Fig. 1, element 106, 110, 130, annotation server 120, col 
2/lines 41-59, col 4/lines 20-31), where an identifier, identifiers the media content 
associated with new annotation content (col 8/lines 5-12). 

It would have been obvious to one ordinary skilled in the art at the time the 
invention was made to modify Hou's system with means where an annotation server, 
coupled to the client computer to maintain an annotation database and to manage 
streaming the multimedia content to the client computer as taught by Sidana, motivation 
would be enable the user via multimedia content identifiers to select for viewing or not a 
annotation included in a document, access a media server to obtain multimedia content 
corresponding to the content identifier which may further be presented in hierarchical for, 
where and annotation set identifier one or more sets of annotations. 

Regarding claim 4, the combined teachings of Hou and Sidana as discussed above, 
wherein the client computer is further present an electronic mail message including a 
multimedia content identifier to a user (Hou: annotations identifiers col 6/lines 61-66, col 
7/lines 1-3, col 8/lines 61-63, col 9/lines 9-14), receive a user selection of the multimedia 
content identifier, access a media server to obtain the multimedia content (Sidana: col 
7/lines 62-col 8/line 26, Hou: col 1 1 /lines 26-34, playback the multimedia content to the 
user). 

Regarding claim 5, the combined teachings of Hou and Sidana as discussed above, 
further transmit, to the media server, an identifier of a temporal segment of the 
multimedia content, and wherein the media server is to stream to the client computer the 
multimedia content beginning with the identified temporal segment (Hou: col 6/lines 61- 
66, col 7/lines 1-3, col 8/line 61-63, col 9/lines). 

Regarding claim 13, 16, and 26, the combined teachings of Hou and Sidana as discussed 
above, the method/program (Sidana: col 1 /lines 55-62) teach receiving data for a new 
annotation corresponding to a temporal range (Hou: col 6/line 61-66, col 7/lines 1-3, col 
8/lines 61-63, col 9/lines 9-14) of media content (Hou: col 1 /lines 39-44, col 3/lines 9- 
13); generating an electronic mail message including both the content of the new 
annotation and an identifier of the media content (Hou: create electronic mail messages 
col 3/lines 8-13, create means: col 2/lines 25-37, col 3/lines 9-13, via interface means to 
receive input data regarding new annotation corresponding to media content col 6/lines 
26-51, containing media content identifiers such as temporal range markers col 6/line 61- 
66, col 7/lines 1-3, col 8/lines 61-63, col 9/lines 9-14, col 1/lines 39-44); and forwarding 
the electronic mail message to a recipient identified by the data (Hou: sending/displaying 
mail: col 1 1/lines 22-25, col 6/lines 14-16 including annotations col 1 1/lines 3-25, Fig. 



1). 
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Regarding claim 14, the combined teachings of Hou and Sidana as discussed above, 
wherein the generating further comprises including, in the electronic mail message, an 
identifier of the new annotation (Sidana: col 8/lines 5-12). 

Regarding claim 15, the combined teachings of Hou and Sidana as discussed above, 
wherein the generating further comprises including, in the electronic mail message, an 
identifier of the temporal range of media content (Hou: col 8/lines 61-63, col 9/lines 1-4, 
col 6/lines 61-66, col 7/lines 1-3). 

Regarding claim 27, the combined teachings of Hou and Sidana as discussed above, 
wherein the electronic mail message further includes a unique identifier of the new 
annotation (Sidana: col 7/lines 62-col 8/line 26). 

Regarding claim 28, the combined teachings of Hou and Sidana, wherein the electronic 
mail message further includes an identifier of one or more annotation sets that the new 
annotation is associated with (Hou: col 5/lines 10-28). 

Regarding claim 31, the combined teachings of Hou and Sidana as discussed above, a 
method comprising: receiving an electronic mail notification of an annotation 
corresponding to media content (Hou: col 6/line 61-66, col 7/lines 1-3, col 8/lines 61-63, 
col 9/lines 9-14) of media content col 1 /lines 39-44, col 3/lines 9-13, receiving an 
electronic mail message including both the content of the new annotation and an 
identifier of the media content create electronic mail messages col 3/lines 8-13, col 
2/lines 25-37, containing media content identifiers/temporal range markers col 6/line 61- 
66, col 7/lines 1-3, col 8/lines 61-63, col 9/lines 9-14, col 1/lines 39-44); and replying to 
the electronic mail notification to generate a new annotation corresponding to the media 
content (Hou: reply means: col 2/lines 31-37). 

Regarding claim 32, the combined teachings of Hou and Sidana as discussed above, 
wherein the replying includes: obtaining, from the electronic mail notification 
(Goodhand: col 1/lines 26-40), an identifier of the annotation; and including the 
identifier of the annotation in the reply (Hou: col 5/lines 10-28). 

Regarding claim 37, the combined teachings of Hou and Sidana as discussed above, one 
or more computer-readable media having stored thereon a computer program that, when 
executed by one or more processors, causes the one or more processors to perform 
functions including: receiving an electronic mail message; and extracting both 
annotation content and annotation identification information from the electronic mail 
message (Hou: col 1 1/lines 26-34). 

Regarding claim 38, the combined teachings of Hou and Sidana as discussed above, 
wherein the computer program further causes the one or more processors to perform 
functions including: creating a new annotation based on the extracted annotation content 
and the annotation identification information (Hou: col 5/lines 5-28); and adding the new 
annotation to an annotation database (Hou: add the new annotations col 9/lines 15-32, 
col 1/lines 17-20, adding annotations means: col 5/lines 29-52, report: col 6/lines 10-14, 
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Fig 3, saving in database) to the annotation database, Sidana software program 
implementation: col 1/lines 55-62). 

fe*arrti« c/a/m 39, the combined teachings of Hou, and Sidana as discussed above 
wherein the extracting annotation identification information comprises extracting from 
the email message: an identifier of media content to which the annotation content 
corresponds and of a temporal segment (Hou: identifier of a temporal segment of the 
multimedia content, and wherein the media server is to stream to the client computer the 
multimedia content beginning with the identified temporal segment, col 6/lines 61-66, col 
7/hnes 1-3, col 8/hne 61-63, col 9/lines), corresponding to the annotation content, of the 
media content; and an identifier of an annotation set that a new annotation including the 
extracted annotation content is to be part of (Row. col 5/lines 5-28). 

4. Claims 6-12, 17-24 and 33-36 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hou et. al. (Hou) U.S. Patent No. 5,838313 in view of Sidana U.S. 
Patent No. 6,081,829 in further view of Goodhand et. al. (Goodhand) U.S. Patent No. 
5,923,848. 



Regarding claim 6, and 12, the combined teachings of Hou and Sidana as discussed 
above, method comprising presenting, to a user, a user interface (Hou: Fig 5 col 2/lines 
62-col 3/lme 14, canvas, col 4/lines 32-col 5/lines) allowing the user to create a new 
annotation (Hou: col 2/lines 52-56, adding/generating means: col 6/lines 10-37) 
corresponding to media content (Hou: Fig. 5); and including, as part of the user interface 
(Hou: abstract: col 1/lines 65-col 2/line 5), afield via which the user can identify a 

7/v II t? lS -° m electronic mail notification of the new annotation (Hou: col 

4/Iines 26-32, Fig. 4, 12, Sidana software program implementation: col 1/lines 55-62) 

However Hou nor Sidana explicitly teach where field via which the user' can 
identify a recipient that is to receive an electronic mail of the new annotation consisting 
of particularly an electronic mail notification 

Electronic mail notification is deemed an inherent feature in the combined 
teachings of Hou and Sidana, where Hou's electronic mail system comprises a standard 
e-mail system and Sidana's annotation media server has also e-mail capabilities e-mail 
systems as know in the art have means for notifying the recipient of the arrival of 
incoming mail, as taught by Goodhand, disclosing where field via which the user can 
identify a recipient that is to receive an electronic mail of the new annotation consisting 
of particularly an electronic mail notification (col 1/lines 26-40), disclosing an 
improvement over existing standard e-mail system (col 8/lines 37-58), comprising means 
for a client computer to present/display electronic mail messages including annotations 
(attachments) to recipients identified by the client computer, including as part of the a 
user interface, a field via which the user can identify a recipient that is to receive an 
electronic mail message and notification comprising a new annotation (col 2/lines 18-25 
46-53, col 5/lines 1-11, col 60/lines 67, col 16/lines 40-61), computer-readable memories 
containing a computer program that is executable by a processor to perform the method 
(col 39/lmes 66-col 40/line 14). 



) 
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It would have been obvious to one ordinary skilled in the art at the time the 
invention was made to modify the combined teachings of Hou and Sidana as discussed 
with means for field via which the user can identify a recipient that is to receive an 
electronic mail notification of the new annotation, as essentially taught by Goodhand, 
motivation would be t further enhance existing e-mail system with means for enabling 
recipient names to be resolved while user is composing a e-mail message without user 
intervention, enabling user to select from a list field adding the notification means level 
of urgency of the received message. 

Regarding claim 7, the combined teachings of Hou, Sidana and Goodhand as discussed 
above, further comprising forwarding the new annotation to an annotation server to 
transmit the electronic mail notification of the new annotation to the recipient (Hou: col 
4/lines 26-31, Fig. 4). 

Regarding claim 8, the combined teachings of Hou, Sidana and Goodhand as discussed 
above, further comprising forwarding the new annotation to an annotation server to be 
added to an annotation set corresponding to the media content (Hou: add the new 
annotations, col 9/lines 15-32, col 1 /lines 17-20, adding annotations means: col 5/lines 
29-52, report: col 6/lines 10-14, Fig. 3, saving in database). 

Regarding claim 9, the combined teachings of Hou, Sidana and Goodhand as discussed 
above, further comprising: allowing the user to identify, via the field of the user interface, 
a plurality of recipients to receive the electronic mail notification (Goodhand: col 1/lines 
36-40, col 1 6/lines 40-61, col 17/lines 6-37); and wherein the forwarding comprises 
forwarding, to each of the plurality of recipients, the electronic mail notification of the 
new annotation (Hou: col 4/lines 26-31, Fig. 4). 

Regarding claim 10, the combined teachings of Hou, Sidana and Goodhand as discussed 
above, further comprising: allowing the user to input, via the field of the user interface, a 
group identifier (Goodhand: identifiers set of recipients (list): col 43/line 5-col 44/line 18, 
Hou: col 4/lines 26-31); identifying an electronic mail address for each of a plurality of 
recipients corresponding to the group identifier; and wherein the forwarding comprises 
forwarding, to each of the plurality of recipients (Hou: col 4/lines 26-31, Fig. 4), the 
electronic mail notification of the new annotation. 

Regarding claim 11, the combined teachings of Hou, Sidana and Goodhand as discussed 
above, further comprising: presenting, as part of the user interface, a default set of 
recipients to receive the electronic mail notification (Goodhand: identifiers set of 
recipients (list): col 43/line 5-col 44/line 18, Hou: col 4/lines 26-31). 

Regarding claim 17, 18, 20-21, and 23, the combined teachings of Hou, Sidana and 
Goodhand as discussed above, receiving an electronic mail notification of a new 
annotation corresponding to media content, the media content having a plurality of 
temporal segments (Hou: col 6/line 61-66, col 7/lines 1-3, col 8/lines 61-63, col 9/lines 9- 
14) of media content col 1/lines 39-44, col 3/lines 9-13, receiving an electronic mail 
message including both the content of the new annotation and an identifier of the media 
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content create electronic mail messages col 3/lines 8-13, col 2/lines 25-37, containing 
media content identifiers/temporal range markers col 6/line 61-66, col 7/lines 1-3, col 
8/lines 61-63, col 9/lines 9-14, col 1/lines 39-44); displaying/presenting the electronic 
mail notification to a user (Hou: display annotations means, col 8/lines 43-48, sending 
and displaying mail: col 1 1/lines 22-25); receiving a user input to access the media 
content; and accessing, upon receiving the user input, a media server to stream one of 
the plurality of segments that corresponds to the new annotation to the user (Sidana: 
displaying means ; col 4/lines 14-43, accessing via user input: col 5/lines 58-col 6/line 6, 
col 7/lines 62-col 8/line 26, wherein user input comprises the selection of a URL of the 
media content identifier). 

Regarding claim 19, the combined teachings of Hou, Sidana and Goodhand as discussed 
above, wherein the presenting comprises: displaying annotation content for the new 
annotation to the user; and displaying at least one identifier to the user, the at least one 
identifier including one or more of a content identifier that identifies the media content; a 
range identifier that identifies a temporal range of the segment corresponding to the new 
annotation (Sidana: in the electronic mail message, an identifier of the new annotation, 
col 8/lines 5-12, an identifier of the temporal range of media content Hou: col 8/lines 61- 
63, col 9/lines 1-4, col 6/lines 61-66, col 7/lines 1-3), an annotation identifier that 
identifies the new annotation; and an annotation set identifier that identifies one or more 
annotation sets that the new annotation is part of (Hou: col 5/lines 10-28). 

Regarding claim 22, the combined teachings of Hou, Sidana and Goodhand as discussed 
above, wherein the receiving a user input comprises receiving a user selection of an 
identifier of the annotation (Hou: col 6/lines 61-66, col 7/line 1-3). 

Regarding claim 24, the combined teachings of Hou, Sidana and Goodhand as discussed 
above, one or more computer-readable media as recited in claim 23, wherein the user- 
selectable identifier comprises a uniform resource locator (URL) that identifies a server 
and a location at the server where the media content is located (Sidana: col 7/lines 62-col 
8/line 26). 

Regarding claim 33, the combined teachings of Hou, Sidana and Goodhand as discussed 
above, teach means/method comprising: receiving an electronic mail thread including a 
plurality of electronic mail messages (Goodhand: col: 4/lines 1-11, group of messages 
and replies related by topic, means for organizing received mail thread: col 4/lines 40- 
43); creating an annotation from at least one of the electronic mail messages (Hou: 
generating an electronic mail message including both the content of the new annotation 
and an identifier of the media content, create electronic mail messages col 3/lines 8-13, 
create means: col 2/lines 25-37, col 3/lines 9-13); and adding the annotation to an 
annotation database (Hou: add the new annotations, col 9/lines 15-32, col 1/lines 17-20, 
adding annotations means: col 5/lines 29-52, report: col 6/lines 10-14, Fig. 3, saving in 
annotation database). 



Regarding claim 34, the combined teachings of Hou, Sidana and Goodhand as discussed 
above, wherein the creating comprises creating a single annotation including each of the 
plurality of electronic mail messages (Hou: generating an electronic mail message 
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including both the content of the new annotation and an identifier of the media content, 
create electronic mail messages col 3/lines 8-13, create means: col 2/lines 25-37, col 
3/lines 9-13, generating a plurality of electronic messages associated with each recipient, 
Goodhand: set of recipients list: col 43/line 5-col 44/line 18, Hou: col 4/lines 26-31). 

Regarding claim 35, the combined teachings of Hou, Sidana and Goodhand as discussed 
above, wherein: the creating comprises generating, for each of the plurality of electronic 
mail messages, an annotation (Hou: generating an electronic mail message including 
both the content of the new annotation and an identifier of the media content, create 
electronic mail messages col 3/lines 8-13, create means: col 2/lines 25-37, col 3/lines 9- 
13); and the adding comprises adding each of the generated annotations to the 
annotation database (Hou: add the new annotations, col 9/lines 15-32, col 1/lines 17-20, 
adding annotations means: col 5/lines 29-52, report: col 6/lines 10-14, Fig. 3, saving in 
annotation database). 

Regarding claim 36, the combined teachings of Hou, Sidana and Goodhand as discussed 
above, further comprising locating, in the electronic mail thread, an identifier of media 
content that the annotation corresponds to (Sidana: mail message further includes a 
unique identifier of the new annotation, col 7/lines 62-col 8/line 26, wherein the 
electronic mail message further includes an identifier of one or more annotation sets that 
the new annotation is associated with, Hou: col 5/lines 10-28). 



Citation of Pertinent Art: 



5 The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure; Copies of documents cited will be provided as set forth in MPEP § 
707.05(a): 

U.S. Patent No. 5,991,365, Pizano et. al. (1 1/23/99) 

U.S. Patent No. 5,893,087, Wlaschin et. al. (04/06/99) 
U.S. Patent No. 5,600,775, King et. al. (02/04/97) 
U.S. Patent No. 5,524, 1 93 Covington et. al. (06/04/96) 
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6 Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Beatriz Prieto whose telephone number is (703) 305- 
0750. The Examiner can normally be reached on Monday-Friday from 6:30 to 4:00 p.m. 
If attempts to reach the examiner by telephone are unsuccessful, the Examiner's 
Supervisor, Mark H. Rinehart can be reached on (703) 305-4815. The fax phone 
number for the organization where this application or proceeding is assigned is (703) 
305-9731. Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is (703) 305- 
3900. 



B. Prieto 
Patent Examiner 
July 24, 2000 
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